feat!: Conceal serde types with Swift SPI #805

jbelkins · 2024-08-29T22:06:30Z

Issue #

Description of changes

Conceals the public interfaces related to serde using the Swift @_spi() annotation.
This makes clear to customers that though public, the serde types are not intended for them to interact with directly.

Types that are now SPI:

SmithyReader, SmithyWriter and all implementations of those types.
All reading & writing closures, and their typealias definitions.
All document reading & writing closures, and their typealias definitions.
The entire SmithyTimestamps module.
A couple middlewares that have serde types in their interfaces.

Also:

The SwiftSymbol factory method is improved with new methods for importing additional symbols and annotating a symbol with more than one SPI.
The serde extension on the ByteStream type is removed from ClientRuntime and moved directly into SmithyReadWrite.

Scope

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

jbelkins · 2024-08-29T22:51:58Z

Package.swift

@@ -103,7 +103,7 @@ let package = Package(
        ),
        .target(
            name: "SmithyReadWrite",
-            dependencies: ["SmithyTimestamps"]
+            dependencies: ["Smithy", "SmithyTimestamps"]


Since ByteStream is now in the Smithy module, SmithyReadWrite takes a dependency on Smithy and provides its own read-write implementation for ByteStream.

jbelkins · 2024-08-29T22:53:38Z

Sources/ClientRuntime/Networking/Http/HTTPResponse+WireDataProviding.swift

@@ -7,9 +7,10 @@

 import class SmithyHTTPAPI.HTTPResponse


Below you will see that many declarations and imports of those declarations are tagged with @_spi(...) to prevent them from showing up as public API.

I will not comment on the rest of these type of changes below.

jbelkins · 2024-08-29T22:54:23Z

Sources/ClientRuntime/Serialization/SerializationUtils/XML/ValueWriters.swift

-import enum Smithy.ByteStream
-
-extension SmithyWriter {
-


The method below was moved into SmithyReadWrite.

jbelkins · 2024-08-30T14:46:02Z

Sources/SmithyReadWrite/SmithyWriter.swift

@@ -47,6 +49,17 @@ public protocol SmithyWriter: AnyObject {

 public extension SmithyWriter {



The function below was moved here from ClientRuntime since ByteStream now resides in Smithy module.

jbelkins · 2024-08-30T15:20:34Z

...rc/main/kotlin/software/amazon/smithy/swift/codegen/events/MessageUnmarshallableGenerator.kt

@@ -206,6 +207,7 @@ class MessageUnmarshallableGenerator(

    private fun renderReadToValue(writer: SwiftWriter, memberShape: MemberShape) {
        val readingClosure = ReadingClosureUtils(ctx, writer).readingClosure(memberShape)
+        writer.addImport(SmithyReadWriteTypes.SmithyReader)


An import of SmithyReader is added here because some of the methods being used are SPI methods extended on that type.

jbelkins · 2024-08-30T15:22:27Z

...re/amazon/smithy/swift/codegen/integration/httpResponse/HTTPResponseBindingErrorGenerator.kt

@@ -83,6 +85,7 @@ class HTTPResponseBindingErrorGenerator(
                        .map { ctx.model.expectShape(it) as StructureShape }
                        .toSet()
                        .sorted()
+                    writer.addImport(SwiftSymbol.make("ClientRuntime", null, SwiftDependency.CLIENT_RUNTIME, "SmithyReadWrite"))


An import of all of ClientRuntime with the SmithyReadWrite SPI annotation is required to access the HTTPResponse.data() instance method.

jbelkins · 2024-08-30T15:23:19Z

...mazon/smithy/swift/codegen/integration/httpResponse/HTTPResponseBindingErrorInitGenerator.kt

@@ -50,6 +52,8 @@ class HTTPResponseBindingErrorInitGenerator(
                    errorShape,
                ) {
                    if (needsReader) {
+                        writer.addImport(SmithyReadWriteTypes.SmithyReader)
+                        writer.addImport(ctx.service.readerSymbol)


This import ensures that all the SPI methods on the reader are exposed.

jbelkins · 2024-08-30T18:54:52Z

...y-swift-codegen/src/main/kotlin/software/amazon/smithy/swift/codegen/SwiftImportContainer.kt

@@ -14,22 +14,19 @@ class SwiftImportContainer : ImportContainer {
    fun addImport(
        packageName: String,
        isTestable: Boolean = false,


Below, the import container is modified to handle more than one SPI name.

jbelkins · 2024-08-30T18:55:58Z

smithy-swift-codegen/src/main/kotlin/software/amazon/smithy/swift/codegen/SwiftWriter.kt

@@ -104,7 +105,7 @@ class SwiftWriter(
    fun addImport(symbol: Symbol) {
        symbol.references.forEach { addImport(it.symbol) }
        if (symbol.isBuiltIn || symbol.isServiceNestedNamespace || symbol.namespace.isEmpty()) return


Below, SwiftWriter is modified to handle symbols that have multiple SPI names.

jbelkins · 2024-08-30T18:56:34Z

smithy-swift-codegen/src/main/kotlin/software/amazon/smithy/swift/codegen/SwiftWriter.kt

        }
        symbol.dependencies.forEach { addDependency(it) }
+        val additionalImports = symbol.getProperty("additionalImports").getOrElse { emptyList<Symbol>() } as List<Symbol>
+        additionalImports.forEach { addImport(it) }


Additional imports are read from the symbol, and each is imported itself.

jbelkins · 2024-08-30T19:13:51Z

...re/amazon/smithy/swift/codegen/integration/httpResponse/HTTPResponseBindingErrorGenerator.kt

@@ -83,6 +85,7 @@ class HTTPResponseBindingErrorGenerator(
                        .map { ctx.model.expectShape(it) as StructureShape }
                        .toSet()
                        .sorted()
+                    writer.addImport(SwiftSymbol.make("ClientRuntime", null, SwiftDependency.CLIENT_RUNTIME, emptyList(), listOf("SmithyReadWrite")))


This import is needed to expose the .data() extension on HTTPResponse.

jbelkins · 2024-08-30T19:15:46Z

...e/amazon/smithy/swift/codegen/integration/httpResponse/HTTPResponseBindingOutputGenerator.kt

@@ -60,6 +62,7 @@ class HTTPResponseBindingOutputGenerator(
                        writer.write("return \$N()", outputSymbol)
                    } else {
                        if (needsAReader(ctx, responseBindings)) {
+                            writer.addImport(SwiftSymbol.make("ClientRuntime", null, SwiftDependency.CLIENT_RUNTIME, emptyList(), listOf("SmithyReadWrite")))


Like on the error init generator above, this import exposes .data() on HTTPResponse.

jbelkins · 2024-08-30T19:16:58Z

.../software/amazon/smithy/swift/codegen/integration/serde/member/MemberShapeDecodeGenerator.kt

@@ -118,10 +119,11 @@ open class MemberShapeDecodeGenerator(
        val memberTimestampFormatTrait = memberShape.getTrait<TimestampFormatTrait>()
        val swiftTimestampFormatCase = TimestampUtils.timestampFormat(ctx, memberTimestampFormatTrait, timestampShape)
        return writer.format(
-            "try \$L.\$L(format: \$L)",
+            "try \$L.\$L(format: \$N\$L)",


The TimestampFormat type is explicitly referenced so it gets the needed SPI import.

The same thing happens below where a timestamp format is rendered.

jbelkins · 2024-08-30T19:18:06Z

...egen/src/main/kotlin/software/amazon/smithy/swift/codegen/swiftmodules/ClientRuntimeTypes.kt

@@ -23,7 +25,7 @@ object ClientRuntimeTypes {
        val ContentLengthMiddleware = runtimeSymbol("ContentLengthMiddleware", SwiftDeclaration.STRUCT)
        val ContentTypeMiddleware = runtimeSymbol("ContentTypeMiddleware", SwiftDeclaration.STRUCT)
        val ContentMD5Middleware = runtimeSymbol("ContentMD5Middleware", SwiftDeclaration.STRUCT)
-        val DeserializeMiddleware = runtimeSymbol("DeserializeMiddleware", SwiftDeclaration.STRUCT)
+        val DeserializeMiddleware = runtimeSymbol("DeserializeMiddleware", SwiftDeclaration.STRUCT, emptyList(), listOf("SmithyReadWrite"))


DeserializeMiddleware and BodyMiddleware are marked SPI because they reference SPI types in their interfaces.

jbelkins · 2024-08-30T19:20:23Z

...egen/src/main/kotlin/software/amazon/smithy/swift/codegen/swiftmodules/ClientRuntimeTypes.kt

-    SwiftDependency.CLIENT_RUNTIME,
-    null,
-)
+    object Composite {


These two symbols are "composite symbols", meaning that they reference other symbols in their definition, in this case the two symbols are arrays where another symbol is the element type.

The new "additionalImports" property on SwiftSymbol is used to ensure the inner types get imported as needed.

jbelkins · 2024-08-30T19:20:58Z

...egen/src/main/kotlin/software/amazon/smithy/swift/codegen/swiftmodules/ClientRuntimeTypes.kt


-private fun runtimeSymbolWithoutNamespace(name: String, declaration: SwiftDeclaration? = null): Symbol = SwiftSymbol.make(
+private fun runtimeSymbol(


This method is modified to allow for multiple SPIs on a symbol, and to allow for additional imports.

jbelkins · 2024-08-30T19:21:53Z

...egen/src/main/kotlin/software/amazon/smithy/swift/codegen/swiftmodules/SmithyFormURLTypes.kt

@@ -9,12 +9,17 @@ import software.amazon.smithy.swift.codegen.SwiftDeclaration
 import software.amazon.smithy.swift.codegen.SwiftDependency

 object SmithyFormURLTypes {
-    val Writer = runtimeSymbol("Writer", SwiftDeclaration.CLASS)
+    val Writer = runtimeSymbol("Writer", SwiftDeclaration.CLASS, listOf(SmithyReadWriteTypes.SmithyWriter))


FormURLWriter is made SPI, and imports the SmithyWriter protocol along with itself.

jbelkins · 2024-08-30T19:22:41Z

...codegen/src/main/kotlin/software/amazon/smithy/swift/codegen/swiftmodules/SmithyJSONTypes.kt

-    val Writer = runtimeSymbol("Writer", SwiftDeclaration.CLASS)
-    val Reader = runtimeSymbol("Reader", SwiftDeclaration.CLASS)
+    val Writer = runtimeSymbol("Writer", SwiftDeclaration.CLASS, listOf(SmithyReadWriteTypes.SmithyWriter))
+    val Reader = runtimeSymbol("Reader", SwiftDeclaration.CLASS, listOf(SmithyReadWriteTypes.SmithyReader))


Smithy JSON Reader & Writer are made SPI and import SmithyReader / SmithyWriter along with themselves.

jbelkins · 2024-08-30T19:23:48Z

...en/src/main/kotlin/software/amazon/smithy/swift/codegen/swiftmodules/SmithyReadWriteTypes.kt

+    val SmithyReader = runtimeSymbol("SmithyReader", SwiftDeclaration.PROTOCOL)
+    val SmithyWriter = runtimeSymbol("SmithyWriter", SwiftDeclaration.PROTOCOL)
+    val Document = runtimeSymbol("Document", SwiftDeclaration.ENUM, emptyList())
+    val ReaderError = runtimeSymbol("ReaderError", SwiftDeclaration.ENUM, emptyList())


All SmithyReadWrite symbols are made SPI except for Document and ReaderError.

jbelkins · 2024-08-30T19:26:02Z

...-codegen/src/main/kotlin/software/amazon/smithy/swift/codegen/swiftmodules/SmithyXMLTypes.kt

-    val Writer = runtimeSymbol("Writer", SwiftDeclaration.CLASS)
-    val Reader = runtimeSymbol("Reader", SwiftDeclaration.CLASS)
+    val Writer = runtimeSymbol("Writer", SwiftDeclaration.CLASS, listOf(SmithyReadWriteTypes.SmithyWriter))
+    val Reader = runtimeSymbol("Reader", SwiftDeclaration.CLASS, listOf(SmithyReadWriteTypes.SmithyReader))


SmithyXML Reader & Writer import SmithyReader & SmithyWriter along with themselves.

Also, all SmithyXML types are made SPI.

jbelkins · 2024-08-30T19:26:57Z

...ift-codegen/src/main/kotlin/software/amazon/smithy/swift/codegen/swiftmodules/SwiftSymbol.kt

@@ -12,11 +12,13 @@ class SwiftSymbol {
            name: String,
            declaration: SwiftDeclaration?,
            dependency: Dependency?,
-            spiName: String?
+            additionalImports: List<Symbol>,
+            spiNames: List<String>,


The factory method in SwiftSymbol adds parameters for additional imports, and changes the spiName import from an optional String to a list of Strings.

jbelkins · 2024-08-30T19:27:44Z

smithy-swift-codegen/src/test/kotlin/StructEncodeGenerationTests.kt

@@ -95,12 +95,12 @@ extension TimestampInputInput {

    static func write(value: TimestampInputInput?, to writer: SmithyJSON.Writer) throws {
        guard let value else { return }


Below, many codegen tests have changed to expect an explicit type on timestamp format.

jbelkins added 6 commits August 29, 2024 14:33

Add SPI to SmithyReadWrite & SmithyTimestamps symbols

c65802e

Add SPI markings to Swift runtime files

160b4cd

SDK builds & tests

d604508

Fix protocol & unit tests

0958d29

Fix lint

516bf67

Remove ByteStream extension in ClientRuntime

e1621de

jbelkins commented Aug 29, 2024

View reviewed changes

jbelkins commented Aug 30, 2024

View reviewed changes

Add new SwiftSymbol features

ab14ee8

jbelkins commented Aug 30, 2024

View reviewed changes

jbelkins added 2 commits August 30, 2024 13:57

Fix ktlint

34578e9

Fix codegen tests

bb4bfef

jbelkins commented Aug 30, 2024

View reviewed changes

jbelkins changed the title ~~Jbe/smithyreadwrite spi~~ feat!: Conceal serde types with Swift SPI Aug 30, 2024

jbelkins marked this pull request as ready for review August 30, 2024 19:44

Merge branch 'main' into jbe/smithyreadwrite_spi

f8bef3f

jbelkins requested review from sichanyoo and dayaffe August 30, 2024 22:04

dayaffe approved these changes Sep 3, 2024

View reviewed changes

sichanyoo approved these changes Sep 3, 2024

View reviewed changes

Merge branch 'main' into jbe/smithyreadwrite_spi

fcf2d6a

jbelkins merged commit 2936b2f into main Sep 3, 2024
27 checks passed

jbelkins deleted the jbe/smithyreadwrite_spi branch September 3, 2024 22:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat!: Conceal serde types with Swift SPI #805

feat!: Conceal serde types with Swift SPI #805

jbelkins commented Aug 29, 2024 •

edited

Loading

jbelkins Aug 29, 2024

jbelkins Aug 29, 2024 •

edited

Loading

jbelkins Aug 29, 2024

jbelkins Aug 30, 2024

jbelkins Aug 30, 2024

jbelkins Aug 30, 2024

jbelkins Aug 30, 2024 •

edited

Loading

jbelkins Aug 30, 2024

jbelkins Aug 30, 2024

jbelkins Aug 30, 2024

jbelkins Aug 30, 2024

jbelkins Aug 30, 2024

jbelkins Aug 30, 2024

jbelkins Aug 30, 2024

jbelkins Aug 30, 2024

jbelkins Aug 30, 2024

jbelkins Aug 30, 2024

jbelkins Aug 30, 2024

jbelkins Aug 30, 2024

jbelkins Aug 30, 2024

jbelkins Aug 30, 2024

jbelkins Aug 30, 2024

		@@ -47,6 +49,17 @@ public protocol SmithyWriter: AnyObject {

		public extension SmithyWriter {


		private fun runtimeSymbolWithoutNamespace(name: String, declaration: SwiftDeclaration? = null): Symbol = SwiftSymbol.make(
		private fun runtimeSymbol(

		@@ -95,12 +95,12 @@ extension TimestampInputInput {

		static func write(value: TimestampInputInput?, to writer: SmithyJSON.Writer) throws {
		guard let value else { return }

feat!: Conceal serde types with Swift SPI #805

feat!: Conceal serde types with Swift SPI #805

Conversation

jbelkins commented Aug 29, 2024 • edited Loading

Issue #

Description of changes

Scope

Choose a reason for hiding this comment

jbelkins Aug 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jbelkins Aug 30, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jbelkins commented Aug 29, 2024 •

edited

Loading

jbelkins Aug 29, 2024 •

edited

Loading

jbelkins Aug 30, 2024 •

edited

Loading